Coalescent simulation in continuous space:Algorithms for large neighbourhood size

Many species have an essentially continuous distribution in space, in which there are no natural divisions between randomly mating subpopulations. Yet, the standard approach to modelling these populations is to impose an arbitrary grid of demes, adjusting deme sizes and migration rates in an attempt to capture the important features of the population. Such indirect methods are required because of the failure of the classical models of isolation by distance, which have been shown to have major technical flaws. A recently introduced model of extinction and recolonisation in two dimensions solves these technical problems, and provides a rigorous technical foundation for the study of populations evolving in a spatial continuum. The coalescent process for this model is simply stated, but direct simulation is very inefficient for large neighbourhood sizes. We present efficient and exact algorithms to simulate this coalescent process for arbitrary sample sizes and numbers of loci, and analyse these algorithms in detail

Scale-invariance in gravity and implications for the cosmological constant

Recently a scale invariant theory of gravity was constructed by imposing a conformal symmetry on general relativity. The imposition of this symmetry changed the configuration space from superspace - the space of all Riemannian 3-metrics modulo diffeomorphisms - to conformal superspace - the space of all Riemannian 3-metrics modulo diffeomorphisms and conformal transformations. However, despite numerous attractive features, the theory suffers from at least one major problem: the volume of the universe is no longer a dynamical variable. In attempting to resolve this problem a new theory is found which has several surprising and atractive features from both quantisation and cosmological perspectives. Furthermore, it is an extremely restrictive theory and thus may provide testable predictions quickly and easily. One particularly interesting feature of the theory is the resolution of the cosmological constant problem.Comment: Replaced with final version: minor changes to text; references adde

Scale-invariant gravity: Spacetime recovered

The configuration space of general relativity is superspace - the space of all Riemannian 3-metrics modulo diffeomorphisms. However, it has been argued that the configuration space for gravity should be conformal superspace - the space of all Riemannian 3-metrics modulo diffeomorphisms and conformal transformations. Recently a manifestly 3-dimensional theory was constructed with conformal superspace as the configuration space. Here a fully 4-dimensional action is constructed so as to be invariant under conformal transformations of the 4-metric using general relativity as a guide. This action is then decomposed to a (3+1)-dimensional form and from this to its Jacobi form. The surprising thing is that the new theory turns out to be precisely the original 3-dimensional theory. The physical data is identified and used to find the physical representation of the theory. In this representation the theory is extremely similar to general relativity. The clarity of the 4-dimensional picture should prove very useful for comparing the theory with those aspects of general relativity which are usually treated in the 4-dimensional framework.Comment: Replaced with final version: minor changes to tex

The physical gravitational degrees of freedom

When constructing general relativity (GR), Einstein required 4D general covariance. In contrast, we derive GR (in the compact, without boundary case) as a theory of evolving 3-dimensional conformal Riemannian geometries obtained by imposing two general principles: 1) time is derived from change; 2) motion and size are relative. We write down an explicit action based on them. We obtain not only GR in the CMC gauge, in its Hamiltonian 3 + 1 reformulation but also all the equations used in York's conformal technique for solving the initial-value problem. This shows that the independent gravitational degrees of freedom obtained by York do not arise from a gauge fixing but from hitherto unrecognized fundamental symmetry principles. They can therefore be identified as the long-sought Hamiltonian physical gravitational degrees of freedom.Comment: Replaced with published version (minor changes and added references

Deep Residual Policy Reinforcement Learning as a Corrective Term in Process Control for Alarm Reduction: A Preliminary Report

Conventional process controllers (such as proportional integral derivative controllers and model predictive controllers) are simple and effective once they have been calibrated for a given system. However, it is difficult and costly to re-tune these controllers if the system deviates from its normal conditions and starts to deteriorate. Recently, reinforcement learning has shown a significant improvement in learning process control policies through direct interaction with a system, without the need of a process model or the system characteristics, as it learns the optimal control by interacting with the environment directly. However, developing such a black-box system is a challenge when the system is complex and it may not be possible to capture the complete dynamics of the system with just a single reinforcement learning agent. Therefore, in this paper, we propose a simple architecture that does not replace the conventional proportional integral derivative controllers but instead augments the control input to the system with a reinforcement learning agent. The agent adds a correction factor to the output provided by the conventional controller to maintain optimal process control even when the system is not operating under its normal condition

Interpretable Input-Output Hidden Markov Model-Based Deep Reinforcement Learning for the Predictive Maintenance of Turbofan Engines

An open research question in deep reinforcement learning is how to focus the policy learning of key decisions within a sparse domain. This paper emphasizes on combining the advantages of input-output hidden Markov models and reinforcement learning. We propose a novel hierarchical modeling methodology that, at a high level, detects and interprets the root cause of a failure as well as the health degradation of the turbofan engine, while at a low level, provides the optimal replacement policy. This approach outperforms baseline deep reinforcement learning (DRL) models and has performance comparable to that of a state-of-the-art reinforcement learning system while being more interpretable

An algorithm for diagnosing IgE-mediated food allergy in study participants who do not undergo food challenge.

BACKGROUND: Food allergy diagnosis in clinical studies can be challenging. Oral food challenges (OFC) are time-consuming, carry some risk and may, therefore, not be acceptable to all study participants. OBJECTIVE: To design and evaluate an algorithm for detecting IgE-mediated food allergy in clinical study participants who do not undergo OFC. METHODS: An algorithm for trial participants in the Barrier Enhancement for Eczema Prevention (BEEP) study who were unwilling or unable to attend OFC was developed. BEEP is a pragmatic, multi-centre, randomized-controlled trial of daily emollient for the first year of life for primary prevention of eczema and food allergy in high-risk infants (ISRCTN21528841). We built on the European iFAAM consensus guidance to develop a novel food allergy diagnosis algorithm using available information on previous allergenic food ingestion, food reaction(s) and sensitization status. This was implemented by a panel of food allergy experts blind to treatment allocation and OFC outcome. We then evaluated the algorithm's performance in both BEEP and Enquiring About Tolerance (EAT) study participants who did undergo OFC. RESULTS: In 31/69 (45%) BEEP and 44/55 (80%) EAT study control group participants who had an OFC the panel felt confident enough to categorize children as "probable food allergy" or "probable no food allergy". Algorithm-derived panel decisions showed high sensitivity 94% (95%CI 68, 100) BEEP; 90% (95%CI 72, 97) EAT and moderate specificity 67% (95%CI 39, 87) BEEP; 67% (95%CI 39, 87) EAT. Sensitivity and specificity were similar when all BEEP and EAT participants with OFC outcome were included. CONCLUSION: We describe a new algorithm with high sensitivity for IgE-mediated food allergy in clinical study participants who do not undergo OFC. CLINICAL RELEVANCE: This may be a useful tool for excluding food allergy in future clinical studies where OFC is not conducted

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces

Traditional controllers have limitations as they rely on prior knowledge about the physics of the problem, require modeling of dynamics, and struggle to adapt to abnormal situations. Deep reinforcement learning has the potential to address these problems by learning optimal control policies through exploration in an environment. For safety-critical environments, it is impractical to explore randomly, and replacing conventional controllers with black-box models is also undesirable. Also, it is expensive in continuous state and action spaces, unless the search space is constrained. To address these challenges we propose a specialized deep residual policy safe reinforcement learning with a cycle of learning approach adapted for complex and continuous state-action spaces. Residual policy learning allows learning a hybrid control architecture where the reinforcement learning agent acts in synchronous collaboration with the conventional controller. The cycle of learning initiates the policy through the expert trajectory and guides the exploration around it. Further, the specialization through the input-output hidden Markov model helps to optimize policy that lies within the region of interest (such as abnormality), where the reinforcement learning agent is required and is activated. The proposed solution is validated on the Tennessee Eastman process control

Hierarchical Framework for Interpretable and Probabilistic Model-Based Safe Reinforcement Learning

The difficulty of identifying the physical model of complex systems has led to exploring methods that do not rely on such complex modeling of the systems. Deep reinforcement learning has been the pioneer for solving this problem without the need for relying on the physical model of complex systems by just interacting with it. However, it uses a black-box learning approach that makes it difficult to be applied within real-world and safety-critical systems without providing explanations of the actions derived by the model. Furthermore, an open research question in deep reinforcement learning is how to focus the policy learning of critical decisions within a sparse domain. This paper proposes a novel approach for the use of deep reinforcement learning in safety-critical systems. It combines the advantages of probabilistic modeling and reinforcement learning with the added benefits of interpretability and works in collaboration and synchronization with conventional decision-making strategies. The BC-SRLA is activated in specific situations which are identified autonomously through the fused information of probabilistic model and reinforcement learning, such as abnormal conditions or when the system is near-to-failure. Further, it is initialized with a baseline policy using policy cloning to allow minimum interactions with the environment to address the challenges associated with using RL in safety-critical industries. The effectiveness of the BC-SRLA is demonstrated through a case study in maintenance applied to turbofan engines, where it shows superior performance to the prior art and other baselines.Comment: arXiv admin note: text overlap with arXiv:2206.1343

Case Study: Monitoring Sleeping Patterns of a Boy with Duchenne Muscular Dystrophy and his Caregivers

Please see the pdf version of the abstract